Elect. Comm. in Probab. 8 (2003)86-92 



ELECTRONIC 
COMMUNICATIONS 
in PROBABILITY 



EXCITED RANDOM WALK 

ITAI BENJAMINI 

Weizmann Institute, Rehovot 76100, Israel 
email: itai@wisdom.weizmann.ac.il 

DAVID B. WILSON 

Microsoft Research, One Microsoft Way, Redmond, WA 98052, U.S.A. 
email: dbwilson@microsoft.com 

Submitted February 21, 2003 , accepted in final form June 10, 2003 

AMS 2000 Subject classification: 60J10 
Keywords: Perturbed random walk, transience 

Abstract 

A random walk on Z rf is excited if the first time it visits a vertex there is a bias in one direction, 
but on subsequent visits to that vertex the walker picks a neighbor uniformly at random. We 
show that excited random walk on Z d is transient iff d > 1. 

1. Excited Random Walk 

A random walk on Z d is excited (with bias e/d) if the first time it visits a vertex it steps 
right with probability (1 + e)/(2d) (e > 0), left with probability (1 — e)/(2d), and in other 
directions with probability X/(2d), while on subsequent visits to that vertex the walker picks a 
neighbor uniformly at random. This model was studied heavily in the framework of perturbing 
1-dimensional Brownian motion, see for instance |51ll4j and reference therein. Excited random 
walk falls into the notorious wide category of self- interacting random walks, such as reinforced 
random walk, or self-avoiding walks. These models are difficult to analyze in general. The 
reader should consult 0] ^fl El El E] an d especially the survey paper J3j for examples. 
Simple coupling and an additional neat observation allow us to prove that excited random 
walk is recurrent only in dimension 1. The proof uses and studies a special set of points ("tan 
points") for the simple random walk. 

2. Recurrence in Z 1 

It is already known that excited random walk in Z 1 is recurrent, indeed, a great deal more is 
known about it UJ. But for the reader's convenience we provide a short proof. 
On the first visit to a vertex there is probability p > 1/2 of going right and 1 — p of going left, 
while on subsequent visits the probabilities are 1/2. Suppose that the walker is at x > for 
the first time, and that all vertices between and x have been visited. The probability that 
the walker goes to x + 1 before going to is p+ (1 —p){l — 2/(x + l)) = 1 — 2(1 —p)/(x+ 1). 
Multiplying over the x's, we see that the random walk returns to with probability 1. 
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3. Transience in I? 

The simple random walk (SRW) in Z 2 visits about n/ logn points by time n, and if the excited 
random walk (ERW) gets pushed to the right n/ logn times, it would very quickly depart its 
start location and never return. But it is not clear what effect that the perturbations have 
on the number of visited points, and it is not obvious that the excited random walk will visit 
ro/logn distinct points by time n. 

To lower bound the number of points that the excited random walk visits, we couple it with 
the SRW in the straightforward way, and count the number of "tan points" visited by the 
SRW. We define the coupling as follows: if the SRW moves up, down, or right, then so does 
the ERW. If the SRW moves left, then the ERW moves left if it is at a previously visited point, 
and if the ERW is at a new point, it moves either left or right with suitable probabilities. At 
all times, the y-coordinates of the SRW and ERW are identical. 

To explain the concept of a "tan point" , we imagine that the simple random walker leaves 
behind an opaque trail, and that the sun is shining from infinitely far away in the positive 
x-dircction. If the SRW visits a point (x,y) such that no point (x',y) with x' > x has been 
visited, then the sun shines upon (x,y), and this point becomes tanned. Formally, we define 
a tan point for the SRW to be a vertex (x, y) that is visited by the SRW before any point of 
the form (x',y) with x' > x. If the sun shines upon the simple random walker the first time 
it is at (x, y), it is straightforward to check that ERW is at a new point. We will show that 
with high probability there are many tan points (so the ERW visits many new points), and 
that this implies that the ERW is transient. 

The probability that a point [x, y) will be tan follows from some enumerative work of Bousquet- 
Mclou and Schaeffer on random walks in the slit plane 3 . 

Lemma 1. Let r and 9 be the polar coordinates of the point (x,y), i.e. r > 0, < 9 < 2it, 
x — r cos 9, and y = r sin 9. Then 

r , s i / , ^ /l + V / 2sin(6»/2) 
Px[(x, y) is tan] = (1 + o(l)W ^U, (1) 

where the o(l) term goes to as r tends to oo. 

This equation does not explicitly appear in [3] , but all the real work that goes into proving it 
is in 0. In the interest of completeness, we explain how this equation follows from explicit 
results in 0: 

Proof. Let a n be the number of walks of length n that start from (0,0), and avoid the non- 
negative real axis at all subsequent times, and let p x ,y,n denote the probability that a random 
such walk ends at the point (x,y). By reversibility of the random walks, 

SRW started from the point (x, y) first hits the 

Pr 

nonnegative real axis at the point (0, 0) and at time n 
Thus Pr[(x,y) is tan] = Y^=o a n/^ n x Px,y,n- Theorem 1 of [3] gives 

a n ( + _ 1/4 

^ = (1 + ' (1)) 2fW" 
Theorem 21 of 3! considers the endpoint (X n , Y n ) of a random walk started from (0, 0) which 
avoids the nonnegative real axis, and gives the limiting distribution of [X n /^/n, Y n /y/n). This 
limiting distribution morally determines the asymptotics of p x ,y,n — a local limit theorem 
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would determine the asymptotics — and the authors prove a local limit theorem for Y n /y/n 
but not X n / \fn let alone the joint distribution (X n /^/n, Y n /^/n). However, since the ordinary 
random walk has a local limit theorem (on vertices such that x + y = n mod 2), one can take 
the limiting distribution of ^(i- e )n, ^(i~e)n an d then run the walk another en steps; upon 
sending e to sufficiently slowly, one can obtain a local limit theorem version of Theorem 21 
of 0: 

/-, /-.xx 2 r 1 / 2 _ r 2, I 2 x + y = n mod 2 

P x ,v,n = (1 + o l L , 5/4 e r > n sm (9 2) x\ y 

r(l/4) ■nP' i 10 x + y^n mod 2 

when r = Q(^/n) and 9 is bounded away from and 27r. Thus we obtain 



0(r 2 ) 



l + \/2sin(0/2) 



2tt 



Next we check that the terms when n « r 2 or n » r 2 contribute negligibly. We may bound 
fS" x Px,y,n by the probability that the walk survives the first n/2 steps (C^l/n 1 / 4 )) times the 
probability that ordinary SRW for the remaining n/2 steps ends at the point (x, y) (0(l/n)). 
Thus 



4 

When n < r 2 we bound ^ x p x ,y,n by the probability that the walk makes it out to radius 
r/2 without hitting the line (which is Oll/r 1 / 2 ) by |^ Eqn 2.40]) times the probability that 
the walk ends up at (x, y) (distance s > r/2 away) at the end of the remaining m < n steps. 
This latter probability is at most e~ s /( 2m ), and assuming n < r 2 /8, it is upper bounded 

by i±£m e -r 2 /(8n). Thus 

E J x P*,v,« < Oil/r 1 / 2 ) x E Oin-^/W) « l/r 1 ' 2 

n^r 2 n^r 2 



Hence when 9 is bounded away from and 2ir and r is large, the terms when n < r 2 or n > r 2 
contribute negligibly, so the formula follows in this case. But the formula in the limiting case 
when 9 approaches or 27r follows from the case when 9 is bounded away from and 2-7T, so 
the formula is valid simply when r is large enough. □ 

We will use the notation introduced by Knuth where 6(/) denotes an expression which is 
upper-bounded by C x f and lower-bounded by c x /, where c and C are positive constants. 
(By contrast, O(f) denotes an expression for which there is an upper bound of C x /, but not 
necessarily any lower bound.) Using this notation, we may crudely approximate Equation Q 
with 

„ r . . . . (en/r 1 / 2 ) whenx>0 

Pr fx, y) is tan = i ' L ~ (2 

LV l!J) J \e(|y|/r 3 / 2 ) whenx<0. V ; 

Equation ^ is useful, but we need two modifications. For our purposes, it would be better to 
have the probability that a point is tan and that it is reached by time n. Then we would have 
the expected number of tan points by time n, which would lower-bound the expected number 
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of times that the ERW is pushed to the right. But a lower bound on the expected number is 
not quite what we need to prove transience of the ERW; what we'd really like to know is that 
with very high probability, the number of tan points by time n is large. 

In order to get the "with very high probability" part of the statement, it would be convenient 
to be working with independent events. To get this independence, we divide the plane into 
bands of height h = h(n) to be determined later, and we will focus on every other band, say 
the even ones. When the SRW first arrives at an even band, let us count the tan points within 
the band that are encountered before the random walk reaches a different even band. These 
counts are independent for the different bands. After n steps it is likely that order s/ii/h even 
bands have been crossed, so it is likely that the number of tan points dominates a sum of 
9(yn//i) independent random variables. 

Lemma 2. Consider a band of height h, i.e., Z x [tjq, j/o + h— 1]. After the SRW first reaches 
this band, with probability 0(1) the SRW hits 0(/i 3 / 2 ) tan points within the band before leaving 
the enclosing band 7L x [y — h, yo + 2h — 1]. 

For the 6(1) part of this lemma, the following proposition is useful. 
Proposition 3. If X is a real-valued random variable and K[X] > 0, then 

E[X] 2 



Pr [X > |E[X]] > 



4E[X 2 



This proposition is essentially exercise 1.3.8 of Durrett j^j. 

Proof of Lemma\^ Consider a point (x, y) within the band, and the ray [x, oo) x y with (x, y) 
at its tip. Consider the largest circle contained within the enclosing band and centered at the 
tip, and also a small enough disk centered at the tip. From Equation it follows that we 
can take the ratios of the two radii to be 6(1) and have the property that for any point p in 
the left half of the small disk and any point q on the outer circle, 

Pr[SRW p hits ray at tip] > 2Pr[SRW g hits ray at tip], 

where SRWp denotes the simple random walk started at point p, and by "hits ray at tip" we 
mean that the first time the walker hits the ray (x+, y) is at the tip (a;, y). Now 

' SRWp hits ray at tip" 
before leaving circle 
' SRWp hits ray at tip" 
before leaving circle 
' SRWp hits ray at tip" 
before leaving circle 



Pr[SRW p hits ray at tip] = Pr 
Pr[SRW p hits ray at tip] < Pr 
Pr[SRW p hits ray at tip] < Pr 



Pr 



SRWp leaves circle and 



then hits ray at tip 
maxPr[SRW g hits ray at tip] 



1 



Pr[SRWp hits ray at tip] 



Pr 



SRWp hits ray at tip 
before leaving circle 



1 



> - Pr[SRWp hits ray at tip] = 6(1/ dist(p, tip) 



1/2 



The point where the random walk first enters the band will lie within the left half of the small 
disk surrounding Q(h 2 ) such points (x,y). In fact, there are &(h 2 ) such points {x,y) within 
radius h of where the SRW first hits the band. Thus 

# tan points within band before SRW departs enclosing band" 
and within radius h from where SRW arrives in band 



E 



> e(/i 2 )x9(l//i 1/2 ) = 6(/i 3/2 ). 
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Next we need a second moment estimate: 



E 



# tan points within band before SRW departs enclosing band\ ' 
and within radius h from where SRW arrives in band / 



< 2 V"Pr[(a;i,yi) tan] x Pr 



< E ( # tan points within radius h from where SRW arrives in band)' 
= ^ Pr[(a;i,yi) and (x 2 ,?/2) tan] 

x\ ,y\ ,X2 ,fj2 within radius h 

< 2 Pr[(xi,yi) tan and (22,2/2) tan after (xi,yi)] 

x\ ,y\ ,a?2,y2 within radius h 

where by "after" we include the possibility that (£2,2/2) = (21,2/1) 

= 2^ Pr[(xi,yi) tan] x Pr[(x 2 ,J/2) after (xi,yi)\{xi,yi) tan]x 
Pr[(x2,y 2 ) tan|(x 2 ,?/2) after and (21,2/1) tan] 

after {x\,y\), (22,2/2) visited before (ara+, 2fe)| 
(2:2,1/2) after (21,1/1) and (aii, j/i) tan 

= 2^Pr[(xi,t/i) tan] Pr[(x 2 - 21,2/2 - 2/1) tan in SRW ] 

= e(h 3 ) 

Combining these estimates with Proposition^ we see that with at least 6(1) probability there 
are at least 0(/i 3 / 2 ) tan points in the band before the SRW departs the enclosing band. □ 

Theorem 4. With probability 1, for all but finitely n€N, the excited random walk has drifted 
right by a distance of at least 0(n 3 / 4 / log 5 ^ 4 n) at time n. In particular, it is transient. 

Proof. Say that the SRW deals with a band if it reaches that band and then reaches a different 
band of the same parity. Suppose that the random walk starts from an odd band in the middle 
of a group of 4k + 1 bands of height h. Then the probability that it fails to leave the group of 
bands after n = (kh) 2 t steps is exponentially small in t. We will optimize k and t later, but 
we will take t to be large so that with high probability the walk leaves the group of 4fc + 1 
bands, and in particular deals with at least k even bands. 

Rather than run the random walk for exactly n steps, let us run it until it deals with k even 
bands. Then the number of early tan points in the different even bands are independent of one 
another, and each one has a 0(1) chance of being at least 0(/i 3 / 2 ). Except with probability 
cxp(— 0(fc)), the number of tan points in the k even bands will be Q(kh 3 / 2 ). 
Since there is only a exp(— 0(f)) chance that the walk has not dealt with k even bands by 
time n, we find that, except with probability exp(— 0(f)) + exp(— 0(fc)), there are <d(kh 3 / 2 ) 
tan points by time n. To optimize our parameters we take f = k, and then we have n = k 3 h 2 . 
Next we consider the location of the perturbed random walk at time n. Typically the random 
walk diffuses by 0(i/n) = 0(fc 3/2 /i), and drifts right by at least Q(kh 3 ^ 2 ). The probability 
that it diffuses by more than k 2 h is exp(— 0(fc)), and the probability that it drifts less than 
0(fc/i 3 / 2 ) is < exp(-0(fc)). We take k = ©(log n), so that the probability of a bad event is 
< 1/n 2 , which gives us h = ©(n 1 / 2 / log 3 ^ 2 n). Except with probability < 1/n 2 , the excited 
random walk has drifted right by at least 0(n 3 / 4 / log 5 ^ 4 n). In particular this event fails only 
finitely often, so the random walk is transient. □ 
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4. Transience in Z , d > 2 

As with ERW in Z 2 , in 1 d we can couple the ERW with SRW, and then (in continuous time) 
couple the SRW in Z d with the SRW in Z 2 . For each tan point of the SRW in Z 2 there is a 
tan point of the SRW in Z d , so as before the ERW is transient. 

5. Speed 

We have seen that the excited random walk on Z d is transient for d > 2, but does it have 
positive speed? 

Theorem 5. Let X n denote the x-coordinate at time n of the excited random walk on Z d with 
bias e/d. If d> 4, then almost surely liminfn^oo X n /n > 0.659e/d; in particular the speed is 
positive. 

Proof. Project down the x-coordinate of the ERW and d — 4 additional coordinates, and 
consider the resulting SRW on Z 3 . Let R n be the range of the SRW on Z 3 by time n, i.e. the 
number of points visited by time n. Since the SRW is transient, E[i?„]/n — > c where c is the 
escape probability of the SRW. Glasser and Zucker JOj (see also pj) determined this escape 
probability c to be 

327T 3 

c = -= = 0.65946 .... 

% /6r(l/24)r(5/24)r(7/24)r(ll/24) 

For our purposes it is not enough to know E[i?„], what we need is the strong law of large 
numbers for i?„ that was proved by Dvoretzky and Erdos !9;: a.s. R n /n — ► c (see also [2] for 
even stronger results on R n ). Thus for any S > 0, a.s. there are only finitely many n for which 
the ERW has not had (c — 6)n pushes to the right by time n. The theorem then follows from 
the ordinary strong law of large numbers. □ 

It seems intuitive that excited random walk in Z 3 also has positive speed, but we do not see 
a proof. Excited random walk in Z 2 is more delicate, and it is not clear even at an intuitive 
level whether or not the speed is positive, though we believe that by time n it has traveled 
distance at least 6(n/logn). 
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